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Supplementary Report on 
Assessing Labor Market 
Outcomes: A Pilot Study 


n May 2015, the State University of New York (SUNY) Board 
[« Trustees adopted a resolution directing Chancellor Nancy 

Zimpher to develop a plan to make applied learning opportu- 
nities available to SUNY students. While all applied learning ex- 
periences — work-based, service-oriented, or directed research — 
are believed to strengthen the academic performance of students,! 
internships are believed to improve employment outcomes on 
graduation.” Research on the effects of applied work-based learn- 
ing experiences on employment rates, salaries, and levels of job 
satisfaction suggests that such beliefs may be warranted.? How- 
ever, the findings from extant research often lack detailed infor- 
mation useful for program development and evaluation at 
department, campus, or system levels. 

Access to detailed and comprehensive administrative data has 
opened up the possibility to pursue continuous, systematic, and 
more refined assessments of the effects of applied learning. How- 
ever, most studies using such data have not examined the effects 
of applied learning on retention, graduation, and labor market 
outcomes.* To better judge the feasibility of using administrative 
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data to measure such effects, we conducted a pilot study of the ef- 
fects of internships — one form of applied learning — at one 
SUNY campus. Working with experienced analysts in the field 
and New York State Department of Labor (NYSDOL) staff, as well 
as SUNY officials and administrators at the SUNY campus partici- 
pating in this pilot study, we identified and addressed a range of 
design, measurement, and implementation matters. Findings and 
conclusions concerning the effects of internships on academic suc- 
cess (retention and graduation) are presented in the main report, 
Applied Work-Based Learning at the State University of New York: Sit- 
uating SUNY Works and Studying Effects. 

In this supplement, we report findings on the feasibility of us- 
ing extant academic unit records linked to NYSDOL Unemploy- 
ment Insurance (UI) wage records to estimate the effects of 
internships on employment outcomes. Findings are presented and 
discussed, with due attention to the constraints of the pilot study 
itself and considerations in interpretation. 

Overall, we find that the effects of internships on employment out- 
comes can be estimated with linked administrative academic-wage re- 
cords. However, there are important design, measurement, and 
implementation considerations that limit the measures available and the 
strength and usefulness of the findings. An important limitation is 
that access to employment and wage information in the NYSDOL 
UI wage record files is, at present, available to SUNY only as data 
for groups of no less than ten graduates. Access to individual 
wage records, linked to administrative academic records, would 
extend and refine the kinds of analyses that could be carried out 
and, therefore, strengthen the confidence researchers and admin- 
istrators could have in the results. 

A further, major limitation with respect to presently available 
information is incomplete coverage. 

NYSDOL wage record information is limited to graduates 
who take up covered employment in New York State (NYS). Em- 
ployment outcomes for graduates who leave NYS, who are 
self-employed, or otherwise not in covered employment are not 
known. The extent to which internships are associated with more 
or less favorable results overall cannot be estimated. Insofar as 
New York pursues data sharing agreements with other states in 
the region, this limitation can be reduced. With the data presently 
available, analyses can uncover associations of internships with 
likelihood of employment in NYS and level of wages of graduates em- 
ployed in NYS. Those are narrower, but still relevant, measures of 
effects for SUNY and its campuses. 

The measures of effects could be extended if campuses and 
programs had information on students who transfer out (to in- 
clude, eventually, employment outcomes for such students) and 
who continue into graduate studies. If not available for every stu- 
dent or graduate, at least information on registrations and aca- 
demic performance for students moving among SUNY campuses 
might be incorporated in analyses. 
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Academic records might be further extended, to include infor- 
mation collected in a systematic way on student participation in 
internships that do not come under a formal course. Initiatives 
with E-Portfolios might offer useful examples of approaches to 
obtain such information, at once complementary to and richer 
than what may be obtained from online surveys of students or 
graduates.° 

Another limitation is the lack of detailed information on in- 
ternships — beyond course designation — to mark participation 
and credits earned. SUNY, and individual campuses, are develop- 
ing and implementing means to assign to each course a classifica- 
tion according to the nature of the applied learning experience 
provided.¢ But, as noted in the main report, we lack additional de- 
tails on features of internships, such as whether internships are in- 
tegrated within a degree program, structured seminars are 
organized within the internship, faculty are directly involved, em- 
ployers who host interns have ongoing relationships to the pro- 
gram, and whether assessments are geared to cognitive and 
noncognitive objectives. Under SUNY Guidelines, all approved 
applied learning experiences are expected to manifest features 
such as these.” However, to the extent that the features differ 
within and among fields of study, they may be associated with 
differences in employment outcomes as well as retention and 
graduation rates. If so, such findings could target areas for change 
in the organization of and arrangements for internships. 

Even lacking details on internship features, extant administra- 
tive academic unit data linked to UI wage record data could be 
used for the assessment of employment outcomes at program, 
campus, and system level. If UI wage record information were 
routinely appended to administrative academic records, changes 
over time in employment outcomes for internships can be tracked 
against the introduction or adaptation of particular internship fea- 
tures or different levels of participation at program or campus 
level. Department faculty, through other methods, could probe 
more fully what might account for changes (if any) in observed 
employment outcomes. 

Beyond design, measurement, and implementation consider- 
ations, the analyses of employment outcomes yield two main 
findings of interest. One finding is that estimates of the effects of in- 
ternship on employment outcomes are mixed — and, generally, more 
mixed than what we found for effects on retention and graduation as pre- 
sented in the main report. The mixed findings may owe to data limi- 
tations, specifically the use of comparisons of groups of internship 
takers to groups of nontakers, as opposed to student-level analy- 
ses made possible with unit-record data. For analyses of the ef- 
fects of internship taking on retention and graduation in the main 
report, we relied on mulitivariate statistical techniques as well as 
propensity score matching. Or, the mixed results may simply re- 
flect the reality of distinct labor markets for graduates from differ- 
ent fields. Labor market outcomes also reflect near-term dynamics 
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in national and regional economies, having greater or lesser ef- 
fects by sector or occupation. We do not fully account for such 
influences. 

The second finding is that some results appear to be highly sugges- 
tive. In particular, we find generally consistent evidence that gradu- 
ates in business with internships were more likely to be employed 
in NYS, more likely to find jobs in NYS sooner, and more likely to 
receive higher wages in NYS jobs than their peers without intern- 
ships. We find less consistent evidence of differences in effects by 
personal attributes of graduates. We judge all results to be sugges- 
tive. Even for business graduates, not all of the differences associ- 
ated with internship taking are statistically significant. In a few 
comparisons, for graduates from other fields or manifesting partic- 
ular demographic or socioeconomic attributes, the pattern is re- 
versed. Moreover, as already noted, the coverage of postgraduation 
experiences is incomplete and the measures of employment out- 
comes are limited to the period immediately following graduation. 
Finally, we don’t know enough to conclude that the internships — 
let alone any particular features of the internships — account for 
observed results. 


1. Design, Measurement, and Implementation 

Administrative data, already collected by campuses and NYS, 
provide an opportunity to measure the impact of internship that 
may be utilized by campuses or systems. An important aim of the 
pilot study was to explore how far administrative academic unit 
records, linked to UI wage records, can be used to assess the ef- 
fects of internships on employment outcomes. While such data in- 
creasingly are being accessed and used for studies of differences 
in employment rates and salaries by field for institutions and 
states, the pilot study is distinctive in its use of linked academic- 
wage records to uncover the effects of a single program feature — 
internship — within and across fields. We sought to explore the 
feasibility of such an approach, provide details on the measures 
available, and identify limitations and gaps. Although undertaken 
for one campus, the considerations and findings from this pilot 
study reveal the potential for, and limitations of, such an 
approach for other campuses and for SUNY. 

General design, measurement, and implementation matters 
are addressed in the main report. We summarize and extend that 
discussion here, drawing out considerations in using linked 
academic-wage record data to assess the effects of internships on 
employment outcomes. 


A. Design 

In approach, the employment outcomes of participants in ap- 
plied work-based learning experiences need to be compared to the 
employment outcomes of graduates who did not have such expe- 
riences in the course of their studies. Graduates in these two 
groups may differ in individual or program attributes that are 
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likely to be associated with their prior decisions to participate in 
internships and also may be associated with employment out- 
comes. So, the method used to uncover the effects of internship 
taking should take into account such attributes. Methods to do 
this aim to address the problem of self-selection into internshps as 
well as to improve the accuracy of the estimation of the effects of 
the internship alone. 

As developed in the main report, we judged a “matched” stu- 
dents strategy as the most appropriate method for the pilot 
study.’ Under such an approach, academic and socioeconomic in- 
formation is used to “match” students (here, graduates), i.e., to 
identify those who are similar in the likelihood of participation in 
an internship. Employment effects of internships are then esti- 
mated by comparing the outcomes of those who participated in 
such experiences with those in the “matched” group. However, 
for the analysis of employment outcomes, we lacked access to the 
unit records needed in order to develop the “matches.” For this 
part of the pilot study, we could not implement the preferred 
method. While we had access to academic unit records for the pi- 
lot study and individual wage information could be linked to the 
academic unit record, only group data with linked academic-wage 
record information were made available to us. As a result, we 
worked with 176 groups of graduates, reducing to eighty-eight 
“pairs.” Each group within a “pair” has in common individual or 
program attributes but differs in participation in internships. We 
discuss this approach further under implementation consider- 
ations below. 

Four considerations for the design of the pilot limit the analy- 
ses and bear upon interpretation. 


@ Many work-based learning experiences are not recorded as 
credit-bearing internships. Internship taking in the pilot 
study is obtained from identified coding on the graduate’s 
academic record, specifically through completion of 
courses organized around an internship recognized for ac- 
ademic credit. In those courses, students meet expectations 
for assessment, per program faculty. But, many students 
participate in internships that are not formally recognized 
by academic programs. Since students with such intern- 
ship experiences cannot be identified in available academic 
records, they fall into the comparison group in the analy- 
sis, i.e., those without internships. To the extent that these 
students have applied work-based learning experiences 
similar to those of students whose internships are recog- 
nized for credit in academic programs, the effects of partic- 
ipation in internships will be underestimated. While the 
number of such students is unknown, it is likely that the 
shares differ by individual and program attributes.? 

Nonetheless, the results can provide meaningful in- 
formation at campus and system levels, if interpreted nar- 
rowly as the effects of participating in an internship 


Page 5 www.rockinst.org 


Higher Education 


Rockefeller Institute 


Supplementary Report on Assessing Labor Market Outcomes: A Pilot Study 


recognized by the program/campus as meeting academic 
requirements with respect to learning aims, learning expe- 
riences, and subject to academic oversight. 


The assessment of effects of internship taking on employment 
outcomes needs to take into account the labor markets in which 
graduates seek, and take up, jobs. While graduates may pur- 
sue a range of employment opportunities, most recruit- 
ment for jobs takes place in submarkets that broadly align 
with fields of study. Published analyses of available data 
from both surveys of individuals and households and 
linked academic-wage record information similar to data 
assembled for the present study show higher employment 
rates and higher starting salaries for those with bachelor’s 
degrees in business and applied sciences and lower em- 
ployment rates and starting salaries for bachelor’s degee 
graduates in humanities and fine arts.!° Since our purpose 
is to isolate on the effects of internship taking, analyses by 
field of study allow for some account to be taken of differ- 
ences in the submarkets in which graduates are likely to 
seek employment. 


Fields of study differ in the extent to which students engage in 
internships. In some fields, internships do not figure in 
academic programs, and few students participate in this 
type of applied learning experience. For others, particularly 
professional or licensed fields, practical field-based experi- 
ences may be expected (if not required), assessed, and re- 
corded for every student. In the pilot study, about 3 percent 
of graduates in history had participated in at least one in- 
ternship while every graduate in journalism participated 
in an internship recorded in academic records. For these 
fields and others representing the extremes of very low or 
very high participation in internships, simple comparisons 
between internship takers and nontakers are not possible. 

With this consideration in view, we worked 
closely with officials at the participating campus to iden- 
tify fields in which internships are voluntary and with pro- 
portions of internship takers at neither extreme (very low 
or very high rates of internship taking). Of these fields, we 
further identified those for which the numbers of both in- 
ternship takers and nontakers and of graduates were large 
enough to permit comparisons via “paired” groups. The 
selected fields are Biology, Business, Health Sciences, and 
Psychology. Graduates may have followed different de- 
gree plans/specializations within the field (B.A. or B.S., for 
example). 

While such an approach allows us to examine the ef- 
fects of participation in an internship, it reveals an impor- 
tant limitation for the use of available linked academic- 
wage record data to analyze the effects of internships on 
employment outcomes. For fields in which internships are 
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expected, required, or commonly pursued (and, equally, for 
fields in which none of these considerations apply), future 
analyses will need to be framed differently. Within fields ex- 
pecting or requiring internships, for example, analyses 
might explore differences in employment outcomes by in- 
ternship feature.!! With some exceptions, information of this 
type is not now routinely or systematically collected and en- 
tered into administrative databases.!2 However, in fields or 
on campuses where arrangements for internships have 
changed, continuous and systematic monitoring of trends in 
employment outcomes using presently available linked aca- 
demic-wage record data might provide indications of effects 
associated with such changes. 


mM Academic records at a single campus have incomplete informa- 
tion on the studies of those who transfer in or of those who leave 
to complete degrees elsewhere. By design, the pilot study is 
limited to students who entered as first year undergradu- 
ates and, for employment outcomes, completed degree 
studies at the participating campus. However, motivated 
by student choices and by policy,!° the share of students 
attending more than one college is substantial and such 
students can be tracked, increasingly, through unit re- 
cords.'4 The issue is a general one, beyond the reach of this 
pilot study, but potentially of greater interest for some 
campuses and for evaluation of applied work-based learn- 
ing across SUNY. 


B. Measurement 

For the purposes of the analysis, we use three employment 
outcome measures: employed within NYS in any one of six quarters 
from the date of graduation (counted as employed in the quarter if 
wages exceed the minimum wage threshold); number of quarters 
until the first match (excluded, if wages are less than the minimum 
wage threshold); highest wages recorded in any of the first six quarters 
from the date of graduation (excluded, if the wages are less than the 
minimum wage threshold in all quarters). 

Here, we elaborate on considerations for measurement raised 
in the main report, Applied Work-Based Learning at the State Univer- 
sity of New York: Situating SUNY Works and Studying Effects. 


m The wage records provide information on formal employment 
within New York State, excluding graduates who are self- 
employed, employed outside of New York State, or continue in 
graduate programs. NYSDOL obtains from nearly every em- 
ployer a wage record for each employee. Employers file 
wage records each quarter. NYSDOL estimates that 97 per- 
cent of those working in NYS appear in the wage re- 
cords.!5 The proportions of graduates employed in NYS 
who come under covered employment may well be lower, 
e.g., if rates of self-employment and/or federal employ- 
ment are higher. Further, considering all graduates at the 
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campus participating in this pilot study, the share appear- 
ing in wage records is likely to be lower since as many as 
one-third continue on for postgraduate studies instead of 
taking up employment or take up employment outside 
NYS.1¢ The proportions will vary by campus across SUNY, 
whether university center, state-operated college or com- 
munity college, and by proximity to a state border that af- 
fords easier access to employment outside of NYS. For 
these reasons (and others), estimates of the proportions 
employed using UI wage records cannot be interpreted as 
employment rates. 

Nonetheless, when considered with reference to 
the direct contribution to NYS’s workforce, observed dif- 
ferences by internship taking in the proportions of gradu- 
ates employed in NYS, and in the earnings of those so 
employed, are themselves appropriate and relevant — if 
partial — measures of the effects of internships. 


@ More reliable measures of “immediate” employment and wages 
occur within four to six quarters after graduation. We believe 
that the “matched” NYS wage data are accurate, both with 
respect to the matching (via Social Security number) and 
posted wage amounts. 1” 

We understand that employer filings report wages 
paid during the quarter. Graduates who commence a 
full-time post at some point after the beginning of the 
quarter will have recorded wages at a level below the 
full-time equivalent rate. To address the issue, we chose to 
use the highest reported wage in any of the first six quarters 
following the date of graduation. The six quarter period al- 
lows for transition into employment, but is short enough 
to exclude most graduates who undertake and complete 
graduate studies (e.g., a master’s degree, but see “part- 
time employment” below). 

Further, the wages recorded on UI unit records in- 
clude bonuses or other compensation at the time of ap- 
pointment. Since we use the highest reported wage in any 
of the first six quarters following graduation as the mea- 
sure of earnings, bonuses or one-time additional compen- 
sation received in a given quarter give an estimate of pay 
above the rate for the year as a whole. For graduates tak- 
ing up employment within six quarters of graduation, we 
believe that additional compensation at the time of em- 
ployment is likely the greater of the two forms of these 
benefits. If so, such one-time additional compensation 
could occur in a “transition” quarter, i.e., a quarter when 
employment commences during, as opposed to at the be- 
ginning of, the quarter (e.g., employment begins in June, 
reference quarter begins in April; employment begins in 
September, reference quarter begins in July). Recorded 
wages for the “transition” quarter are less likely to be sub- 
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stantially above the wages for the first, full quarter in cov- 
ered employment. 

Finally, we anticipate that effects of internships 
may carry forward, indeed increase, over time. Small or no 
differences in the first postgraduation, six-quarter period 
cannot be taken as evidence that the effects of internships 
on employment outcomes do not appear or grow in five or 
ten years.'® 

Since we are working with three successive cohorts 
of entering students who graduate over a four-year period, 
we use the regional Consumer Price Index for Urban Wage 
Earners and Clerical Workers (CPI-W) to adjust all wages 
to 2014 dollars. Other studies adopt different approaches, 
each seeking to arrive at a better estimate of regular earn- 
ings over a year or for a typical quarter.!° 


= Employment and wages in any quarter may pick up casual or 
part-time employment. For the pilot study, the effect of inter- 
est is employment (and salaries) in a regular, full-time po- 
sition. 

Graduates employed part-time, and notably those 
continuing graduate education with support from an 
assistantship, also appear on wage records. To exclude 
such graduates from the analysis — that is, to exclude 
from counts of employed (and from analyses of wages) 
any graduate other than those most likely to be in regular 
employment, we established a wage threshold correspond- 
ing to the minimum wage in NYS applicable to the rele- 
vant year. Here, too, other studies using wage record 
information adopt a similar approach.” 

However, we note that the threshold, on an annual 
basis, of about $13,000 is below the stipend at many univer- 
sities for a graduate assistant pursuing full-time studies.” 
To the extent such graduates are included in the analyses, 
employment rates will be higher, and mean or median 
wages likely lower, in the fields of study so affected. 


Measures of internship taking and individual and program 
characteristics, taken from administrative academic records at the 
participating campus, are the same as those used in the the part of 
the pilot study concerned with the effects of internship on reten- 
tion and graduation. For the purposes of this pilot study, a gradu- 
ate who took at least one internship for academic credit during 
the course of their degree studies is considered an internship 
taker. As described in the main report, the majority of students 
with internships received credit for internship in more than one 
academic term, and most internships came later in the degree pro- 
gram. All other variables also are categorical: Academic program 
(Biology, Business, Health Sciences, Psychology, Other); gender 
(male/female); racial/ethnic group (White, Asian, Black, His- 
panic, Other); SAT Math score (above median/ below median); 
and tuition residency in first term (NYS/other). 
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C. Implementation 

As mentioned, our work with UI wage record information 
obliged us to follow a sequence of steps in order to conduct the 
study. We elaborate here on the main considerations. 


mB Achieving sufficient numbers for analysis. The initial sample 
consisted of entering cohorts for 2006, 2007, and 2008. 
Across all three cohorts, 8,348 students were included. 
Working with three successive cohorts enabled a sample 
size sufficient to support analyses that met the NYSDOL 
minimum cell count requirements to obtain grouped UI 
wage record data. To allow for six quarters of wage record 
by the end of 2014, we identified graduates up to Summer 
2013, which would be five years from entry for the 2008 co- 
hort. To ensure that graduates across cohorts are alike in 
terms of years from entry to graduation, we limited gradu- 
ates from the 2006 and 2007 cohorts up to Summer 2011 and 
Summer 2012, respectively. We excluded those who had ac- 
cumulated grade point average (GPA) of 2.5 or lower at the 
time of graduation, since most would not have been permit- 
ted to participate in internships. We also excluded, in this 
analysis of employment outcomes, graduates who were 
identified as “non-resident alien.” Further, nineteen gradu- 
ates were dropped because their academic records lacked 
Social Security numbers used for matching. Following these 
rules, we identified 4,488 graduates from the larger cohorts 
for wage record matching. Of those graduates, 2,599 were 
matched in the NYSDOL wage record files covering the des- 
ignated postgraduation period. 


m Identifying fields of study. Analyses of internship taking by 
specific field of study for graduates were carried out by 
Rockefeller Institute staff. About 100 named academic 
fields in which bachelor’s degrees were awarded are repre- 
sented in the sample. As described above, a good number 
of the fields had just a few graduates for the cohorts exam- 
ined, and so were excluded from the list of fields identified 
for separate analysis. Other fields had sufficient numbers 
of graduates, but also as noted above, were distinguished 
by near universal (e.g., professional fields) or very modest 
(e.g., some humanities fields) participation in formal in- 
ternship courses as recorded on transcripts. We chose 
fields that had sufficient numbers of graduates and reason- 
able proportions of internship takers compared to non- 
takers to permit analyses. Graduates in the named fields 
— Biology, Business, Health Science, and Psychology — 
include those following different specializations; in one 
field, the specializations lead to a B.A. or a B.S. Such grad- 
uates, in our judgment, followed broadly similar 
coursework and entered roughly similar job markets. 

Importantly, these decisions were guided by ad- 
vice from officials at the participating SUNY campus who 
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had close knowledge of the study programs concerned 
and the extent to which participation in internships is 
voluntary. 


= Obtaining employment outcome information from UI wage re- 
cords linked to administrative academic data for graduates. Un- 
der implementation of the present Memorandum, 
NYSDOL does not provide linked individual wage re- 
cords. Rather, NYSDOL provides information (cell counts, 
averages, measures of spread) for any group with ten or 
more cases. 

Adopting a conservative estimate that at least half 

of the Social Security numbers submitted for graduates 
would not be matched with NYSDOL’s wage records (for 
graduates not employed in a covered position, employed 
outside of NYS, employed part-time, in graduate study), 
we established 176 groups, identified by codes on the file 
to be submitted to NYSDOL. The groups represent 
eighty-eight “pairs” of groups of graduates for those who 
had internships and those who did not. The “pairs” were 
established by individual or program attributes (field of 
study, gender, racial ethnic group, Pell Grant eligibility, 
SAT Math score, NYS residence for tuition fee purposes). 
Measures of central tendency and spread for each of the 
three employment outcomes were obtained for each group 
within every “pair.” Groups were successively 
disaggregated, by one, two, three, four, and five attributes, 
in general trying to preserve a minimum cell size of 
twenty or greater in the constructed group. Four of the 176 
groups returned a number of matches insufficient to 
permit reporting by NYSDOL. 


™ Carrying out statistical analyses. We undertook statistical 
analyses of differences in proportions employed in NYS, 
quarters to employment in NYS, and mean quarterly 
wages from NYS employment. Such tests are used to pro- 
vide evidence that the differences observed for any of the 
“paired” groups are likely to exist.” The tests take the 
place, in this analysis, of the multivariate techniques used 
for assessment of retention and graduation effects of in- 
ternships as presented in the main report. 

A number of studies using linked academic-wage 
record data rely fully on descriptive statistics of the pro- 
portions employed and median wages (sometimes with as- 
sociated measures of dispersion, such as the interquartile 
range or quartiles or deciles).2 The argument in favor of 
the median as a measure is that the influence of extreme 
values is minimized. Inspection of the descriptive statistics 
on wages reveals median wages are as much as 15 percent 
below the means for groups in which extreme (high) 
wages are recorded.”4 
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The statistical tests allow for a conservative ap- 
proach to the interpretation of differences in employment 
outcomes between intern takers and non-takers. The re- 
sults direct attention to “pairs” where differences in esti- 
mated effects are relatively large. Means and medians (as 
well as other descriptive statistics) are provided in the sup- 
plementary tables, and so can be consulted to examine all 
patterns as well as to identify groups and “pairs” where 
means and medians diverge. 


ll. Analysis and Findings 

We use descriptive data to explore the relationships between 
participation in an internship and employment outcomes, taking 
into account identified academic, demographic, and socioeco- 
nomic backgrounds of graduates. 

Notwithstanding our reference to “effects,” the differences 
may not signal causal relationships. Even if participation in an in- 
ternship is found to be associated with a higher likelihood of em- 
ployment in NYS on graduation or higher NYS earnings, the 
associations may mask other causes for such favorable outcomes. 
For the analysis of retention and graduation effects (discussed in 
the main report), we relied on multivariate regression and pro- 
pensity score matching (PSM) to support arguments of causality. 
While conceptually sound, the methods are limited to the extent 
that the most important observable attributes of graduates that 
determine their probability of internship taking are available. As 
acknowledged in the main report, we lack a number of observable 
measures for the regressions and PSM. We expect that 
unobservable measures (such as, motivation) are also important. 

For the analysis of employment outcomes, we rely on compar- 
isons of means. For the present analysis, we do not have available 
individual unit record data. Rather, we have obtained means (and 
other descriptive statistics) for identified “pairs” of groups, in 
each of which graduates have the same individual and/or pro- 
gram attributes but differ with respect to internship-taking.” 

Supplementary tables in the Appendix provide descriptive 
statistics (Tables S.1 to S.4) and statistical results (Tables 5.5 to 
S.8). 


A. Internship Taking and Employment in NYS 

Overall, some 60 percent of graduates from the participating 
campus took up employment in NYS within six quarters of gradu- 
ation. The proportions vary by field and program and individual 
attribute of the graduate. Details are provided in Supplementary 
Table S.5. 

Overall, internship taking appears to be marginally associ- 
ated with the likelihood of taking a job in NYS within six quar- 
ters of graduation. The overall mean difference, of 1.3 
percentage points, is not statistically significant. For most aca- 
demic, demographic, or socioeconomic attributes, the differences 
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in proportions employed in NYS between internship takers and 
non-takers are relatively small, mostly insignificant, and suggest 
no particular pattern. 

However, larger differences in NYS employment outcomes (as 
picked up in UI wage records) emerges particularly by field. Se- 
lected results are shown in Table 1 below for graduates in busi- 
ness and psychology. Details for other fields and attributes are 
provided in Supplementary Table S.2. 


Table 1. Differences Associated with Internship in the 
Proportions of Graduates Employed in NYS 


All fields Business Psychology 

All graduates 1.3 O:7 ** -5.0 
Male 22. 17.2. *** NA 
Female 0 0.5 -1.8 
White -3.2 -7.5 -16.8 ** 
Asian 2.4 12°* -2.1 
Offered Pell Grant 2.1 15:6: 22% -7.9 
Not Offered Pell Grant 2.1 2.9 -3.1 
SAT Math Above Median 3.4 * 20.2 *** 1.9 
SAT Math Below Median -2.3 -4.3 -9.4 * 

Note: Graduates Employed in NYS are those with wages above the minimum 

wage level in any of the six quarters commencing with the quarter of 

graduation. See text for graduates excluded from the analysis. 

NA Less than ten graduates with matched NYSDOL wage records. 

*0<.10, **p<.05, ***p<.01 


In particular, Business graduates who took internships are 9.1 
percentage points more likely to have a job in NYS than their 
peers who did not participate in internships. The difference is sta- 
tistically significant (p<.05), as is the 17.2 point difference associ- 
ated with internship taking for Business graduates who are male 
(p<.01), the 12.0 point difference for Business graduates who are 
Asian (p<.10), the 15.6 point difference for Business graduates 
who were offered Pell grants in the first term (p<.01), and the 20.2 
point difference for Business graduates with SAT Math scores 
above the median (p<.01). That is, internship taking is associated 
with NYS employment following graduation for Business gradu- 
ates who differ for a number of academic, demographic, and so- 
cioeconomic characteristics. 

In contrast, for Psychology graduates, the 5.0 percentage point 
difference in the proportions employed in NYS is not statistically 
significant. Further, for Psychology graduates with particular aca- 
demic, demographic, or socioeconomic attributes, the effects of in- 
ternship taking were mostly insignificant. For Psychology 
graduates who are White and who have SAT Math scores below 
the median, internship taking appears to be associated with a 
lower likelihood of NYS employment within six quarters immedi- 
ately following graduation (-16.8 percentage points and -9.4 per- 
centage points, respectively, p<.10 or lower). 
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B. Internship Taking and Time to Employment 

Turning only to those who took positions in NYS, the time be- 
tween graduation and employment was an estimated 2.8 quarters, 
on average (median and mode, both two quarters commencing 
with the quarter of graduation). Overall, differences in the time to 
secure a NYS job appear to be little affected by internship taking. 
For all graduates, the mean difference is .03 quarters (roughly two 
days sooner). Details are provided in Supplementary Table S.7. 

However, looking again at results within fields of study, the 
time to NYS employment shows a somewhat larger association 
with internship taking. Selected results are shown in Table 2 
below. Details for other fields and attributes are provided in Sup- 
plementary Table S.8. 


Table 2. Differences Associated with Internship in 


Time to Employment for Graduates Employed in NYS 


All fields Business Psychology 

All graduates -0.029 -0.390 ** 0.268 * 
Male -0.056 -0.537 ** NA 
Female -0.011 0.186 0.171 
White 0.047 -0.467 0.021 
Asian -0.132 -0.354 -0.006 
Offered Pell Grant -0.097 -0.433 * 0.262 
Not Offered Pell Grant 0.034 -0.357 0.264 

SAT Math Above Median 0.009 -0.408 * 0.427 * 
SAT Math Below Median -0.054 -0.421 * -0.054 


Note: Graduates Employed in NYS are those with wages above the minimum 
wage level in any of the six quarters commencing with the quarter of graduation. 
See text for graduates excluded from the analysis. 


NA Less than ten graduates with matched NYSDOL wage records. 


*o<.10, **p<.05, ***p<.01 


In particular, internship takers assumed NYS jobs an esti- 
mated .39 quarters, or five weeks, sooner than non-takers, for Busi- 
ness graduates, but .27 quarters, or 2 weeks later, for Psychology 
graduates (p<.10 or lower). The overall findings suggest the ef- 
fects of internship taking emerge by field, and may reflect pro- 
gram and labor market factors. In fact, the findings provide very 
little evidence that differences in time to NYS employment within 
each field are associated with academic, demographic, or socio- 
economic attributes. The patterns for Business and Psychology 
graduates find broad parallels in time to NYS employment results 
for, respectively, Health Science and Biology graduates. 


C. Internship Taking and Quarterly Wages 
of Graduates Employed in NYS 
For those graduates who took up employment in NYS, quar- 
terly wages (limited to those employed and earning above the 
minimum wage threshold in the reference quarter) differ by field, 
gender, racial/ethnic group, Pell grant eligibility, and SAT Math 
score. In particular, Business graduates, male graduates, 
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graduates who received Pell grants in their first academic term, 
and graduates with SAT Math scores above the median show rela- 
tively higher quarterly wages reported by employers to NYSDOL. 
Simple descriptive statistics are provided in Table S.4. 

No pattern emerges when NYS quarterly wages are compared 
overall for those graduates who had an internship and those with- 
out internships in the course of their degree studies. The overall 
comparisons may reflect, in part, the influence of extreme values 
within wage records. For all graduates employed in NYS follow- 
ing graduation, in the quarter with the highest wage among the 
first six following the date of graduation, internship takers re- 
ceived $9,115 on average (in 2014 dollars). For those without in- 
ternship experiences during their degree programs, the average 
wage was $9,341. Using medians, internship takers recorded quar- 
terly earnings of $8,183 while those who had no internship experi- 
ences showed median earnings of $7,934. 

While there is no evident common pattern when NYS quar- 
terly earnings (mean or median) are compared for each individual 
“paired” group, some differences emerge by field. Selected find- 
ings are presented in Table 3 below. Details for all fields and at- 
tributes are provided in Supplementary Tables S.7 and S.8. 


Table 3. Differences Associated with Internships in 
Quarterly Wages of Graduates Employed in NYS 


All fields Business Psychology 
All graduates -S226 $1,980 *** $272 
Male 330 2,483 ** NA 
Female -384 1,386 ** 488 
White 134 1,430 -610 
Asian -1,522 ** 1,043 * -603 
Offered Pell Grant -441 1,951 *** 1,769 
Not Offered Pell Grant -34 2,050 * -1,096 * 
SAT Math Above Median -478 * 2,285 *** -1,236 ** 
SAT Math Below Median 93 1,640 ** 1,143 


Note: Graduates Employed in NYS are those with wages above the minimum 
wage level in any of the six quarters commencing with the quarter of graduation. 
The highest wages, for any quarter in which wages exceed the minimum wage 
threshold, are used in for the comparisons. See text for graduates excluded from 
the analysis. 


NA Less than ten graduates with matched NYSDOL wage records. 


*p<.10, **p<.05, ***p<.01 


For business graduates, those having had internships recorded 
wages roughly $2,000 (point estimate, $1,980) greater, on average, 
than wages recorded by their peers without internships (signifi- 
cant at p<.01). This difference is on the order of 10 percent, rela- 
tive to average quarterly wages received, within the field. For 
Psychology graduates, the estimated difference in wages of $272 
favoring internship takers is not statistically significant. 
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The relatively strong and favorable results for Business gradu- 
ates who took internships largely holds across academic, demo- 
graphic, and socioeconomic attributes. In particular, although the 
size of the difference in quarterly wages attributed to internships 
for Business graduates varies from $1,000 to $2,500, the effect is 
significant for both males and females, with or without a Pell 
grant offer, or SAT score above or below the median. For Psychol- 
ogy graduates, the association between internship taking and 
quarterly wages from employment in NYS following graduation 
is mostly weak and statistically insignificant. The exceptions ap- 
pear for Psychology graduates with internship experiences who 
were not offered Pell grants or with SAT scores above the median. 
In both instances, the comparisons show graduates with intern- 
ships in this field recorded lower wages (on the order of an esti- 
mated $1,000 in the reference quarter) compared to their peers 
without internship experiences (p<.10 or lower). 

The differences in the patterns of associations between intern- 
ship taking and employment outcomes by field invite closer con- 
sideration of both the orientations of the programs and the 
destinations of graduates. The proportion of Psychology gradu- 
ates with internships is lower than found for graduates in Busi- 
ness, and the proportion of Psychology graduates with and 
without internships employed in NYS following graduation is 
also lower than for their peers in Business. Although not revealed 
in the data available to us, graduates in Psychology may be more 
likely to pursue postgraduate study immediately following grad- 
uation (with employment outcomes manifested later). 


Ill. Conclusions 

To date, assessments of the effects of applied learning in gen- 
eral, and work-based applied learning in the form of internships 
in particular, on employment outcomes are limited. Analyses of- 
ten lack methodological rigor needed to give confidence in the 
findings or details necessary to be useful for program develop- 
ment and evaluation. Growing access to unit records, both admin- 
istrative academic records maintained at campuses and wage 
records submitted by employers under unemployment insurance 
reporting requirements, opens up the possibility to obtain more 
reliable and more useful estimates of such effects. 

This pilot study was undertaken to gauge the feasibility and 
potential value of using linked administrative academic-wage re- 
cord data to assess employment outcomes of work-based applied 
learning experiences offered within SUNY Works. We conclude 
that, subject to limitations that presently apply within SUNY, such 
data can be used to estimate the effects of internships (and, in- 
deed, other forms of applied learning) on employment outcomes. 

The pilot study uncovered findings that are useful, some 
seemingly evident and others suggestive. First, the more impor- 
tant differences in the effects of internships on “immediate” em- 
ployment outcomes in NYS appear to occur by field of study. 
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Here, both features of the internship offered and labor market 
considerations likely come into play. Second, individual attributes 
of gender, racial/ethnic group, Pell grant eligibility, or SAT score 
generally show no patterns for the effects of internship on em- 
ployment outcomes in this limited sample. That finding is gener- 
ally encouraging, in that the effects of internships on employment 
outcomes should apply to all students regardless of background. 
However, to the extent that formal internship taking within de- 
gree programs is associated with more favorable employment 
outcomes for groups otherwise considered less likely to secure 
good jobs at good wages (e.g., females, graduates from 
underrepresented groups, Pell grant recipients), such results 
might be considered a particularly good result. Differences by at- 
tribute do emerge within fields, both favorable and unfavorable, 
and open up questions about why and how personal attributes 
may matter with respect to participation in formal internships in 
particular fields. These findings apply to a limited set of employ- 
ment outcomes, namely the effects of internship on “immediate” 
employment outcomes within NYS. Those employed outside 
NYS, or who take up full-time employment later, are excluded 
from the analysis. 

The limitations warrant consideration for the improvement of 
analyses using linked administrative academic-UI wage record 
data and for the use and interpretation of findings. 

The key limitation is that, under the current Memorandum 
with NYSDOL, SUNY does not have access to unit UI wage re- 
cords. Analyses are possible with grouped data (and we pursued 
that approach), although such analyses necessarily constrain the 
extent of the empirical work that can be carried out.26 

Three other limitations have to do with coverage, classifica- 
tion, and measurement. 

The present study worked with entering cohorts at the partici- 
pating SUNY campus, tracked through to graduation and into 
employment. Students with internship experiences who trans- 
ferred to another college or left higher education were not 
tracked, and students who transferred into the SUNY college and 
participated in internships were excluded. Moreover, only those 
graduates taking up jobs in NYS were picked up in the post- 
graduation period. Finally, students pursuing graduate studies 
during the period, but receiving a stipend above the minimum 
wage threshold, were included in the study. The extent to which 
these gaps in coverage and challenges in sampling matter in the 
assessment of employment outcomes is not known. 

With respect to classification, administrative academic records 
permit the identification of internship taking through registration 
in a formal course under which the internship is organized and 
credit is granted. We lack information on students who may have 
participated in internships that are not recognized under a formal 
course offering. Moreover, administrative academic records lack 
details on the features or nature of the internship experience. The 
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findings from this study are suggestive in this regard: Programs 
in Business (and to a lesser extent in Health Science), with gener- 
ally more favorable effects of internships on employment out- 
comes, may have in common key features, among which are 

(a) established internships as a regular and recognized option in 
the degree programs; (b) ongoing relationships with employers 
who host interns; (c) strong participation in internships; and 

(d) relatively lower rates of postgraduate study. Details confirm- 
ing these kinds of features are known at the program and campus 
level, less so at the system level where administrative academic 
data provide the primary source of information on programs 
within and across campuses. As noted, efforts to identify features 
and attributes of internships (and other forms of applied learning) 
in administrative academic information systems are underway at 
SUNY campuses. As these efforts proceed, closer attention might 
be given to particular features for internships that, when analyzed 
with linked administrative academic-UI wage record data, could 
be useful for program development and evaluation. 

That said, analyses using linked unit record data are most use- 
ful when conceived in the light of a campus profile in terms of de- 
gree programs and circumstances with regard to student intake 
and likely graduate destinations. On the former, the selection of 
fields for analysis in this pilot study was based on close knowl- 
edge and advice from officials at the participating campus, as op- 
posed to application of a standard classification of degree 
programs used by SUNY. Such system-wide classifications can be 
used to good effect for some purposes. For this pilot study of a 
single campus, however, campus-level knowledge led to a stron- 
ger implementation of the design and, arguably, more useful find- 
ings for the programs and campus concerned. 

Lacking details on internship or program features, analyses 
similar to those carried out in the pilot study may still be useful. A 
great advantage of the linked administrative academic-UI wage 
record data is that routine data collection allows for relatively 
easy updating each year or two. Individual programs or campuses 
can make use of successive years of data on program graduates, 
distinguished by participation in internship among other individ- 
ual or program attributes. As changes in internship offerings (e.g., 
organization of the internship, involvement of faculty or employ- 
ers) are implemented, employment outcomes can be tracked from 
before to after such changes. Ongoing monitoring of this type can 
be made more useful when other details (e.g., recording of intern- 
ships not taken for credit; other forms of applied learning) are 
available in the unit records. 

Finally, with respect to measurement, the NYS employment 
outcomes examined in the pilot study reflect two limitations. First, 
the minimum wage threshold used to screen out part-time em- 
ployment is likely too low, not least with respect to stipends paid 
to graduates pursing postgraduate studies. For fields with rela- 
tively large proportions continuing on to advanced degree work, 
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rates of employment will be higher and average wages lower 
when compared to a subset of graduates entering regular, 
full-time employment. A different (higher) threshold can be used, 
although the rationale for any threshold derives from a careful de- 
lineation of the subgroup of graduates of interest. The second lim- 
itation is that the NYS employment outcomes used in this study 
refer only to the “immediate” postgraduation period (up to six 
quarters). As suggested in studies of the time-path of earnings by 
field, favorable outcomes (here, the effects of internships) may 
emerge more strongly in five or ten years. 

In sum, administrative data can be usefully marshalled to 
study the impact of internships on employment opportunities. 
Our analysis of employment outcomes shows mixed results due, 
in part, to data restrictions that limited our analysis to matched 
“pairs,” rather than matched unit records as in the main report. 
And, in part, our results likely reflect differences in the labor mar- 
kets for graduates in different fields. Ideally, reseachers evaluat- 
ing the impact of applied-learning experiences, like internships, 
would have access to unit record data (where available); regional 
employment and academic data to account for graduates who 
leave the state; and comprehensive student data (through univer- 
sity, state, or national systems) to account for students who enter 
a campus as a transfer student or leave a campus to pursue stud- 
ies elsewhere, including graduate study. They would also account 
for differences among academic majors of graduates and among 
the labor markets they are likely to enter. This is an ambitious, but 
worthwhile, agenda. 
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Table S-1. Graduates Employed in NYS by Internship 


Experience, Field of Study, Gender, Racial/Ethnic Group, Pell Grant 
Eligibility, SAT Math Scores, or Tuition Residency in First Term 


Number 

Stratum/Grouping of Cases Percent 
Internship Taking 

At least once 680 26.2 

Never 1,919 73.8 
Field of Study 

Biology 250 9.6 

Business 281 10.8 

Health Science 367 14.1 

Psychology 319 12.3 

Other 1,382 53.2 
Gender 

Male 1,211 46.6 

Female 1,388 53.4 
Racial/Ethnic Group 

American Indian/Alaska Native 5 0.2 

White 858 33.0 

Asian 754 29.0 

Black 192 7.4 

Hispanic 233 9.0 

Race Unknown 557 21.4 
Pell Grant Eligibility 

Offered Pell 1,193 45.9 

Not Offered Pell 1,406 54.1 
SAT Math Score 

Above Median 1,325 51.0 

Below Median 1,274 49.0 
Tuition Residency in First Term 

New York 2,504 96.3 

Other 95 3.7 
Total 2,599 100.0 
Note: Graduates employed in NYS refer to graduates with at 
least one quarter with wages above minimum wage threshold in 
any of the six quarters commencing from the quarter of 
graduation. Graduates refer to those completing bachelor's 
degrees within five years of entry, for each of three cohorts 
entering in 2006, 2007, and 2008. For the analysis, those 
identified as “non-resident alien” are excluded, as are graduates 
with cumulative GPA of 2.5 or below who were unlikely to have 
been permitted to take internships. Nineteen graduates for 
whom no Social Security number was available for a possible 
wage match also were excluded from the analysis. 


Table S-2. Graduates Employed in NYS by Internship Experience 


and Field of Study, Gender, Racial/Ethnic Group, Pell Grant 
Eligibility, SAT Math Scores, or Tuition Residency in First Term 


At Least One Internship 


(Percent) 

Stratum/Grouping No Yes 
Field of Study*** 

Biology 78.8 2152 

Business 65.5 34.5 

Health Science 57.8 42.2 

Psychology 78.7 21.3 

Other 77.8 22.2 
Gender*** 

Male 79.9 20.1 

Female 68.6 31.4 
Racial/Ethnic Group*** 

White 77.6 22.4 

Asian 73.7 26.3 

Black 65.1 34.9 

Hispanic 66.5 33.5 

Other 74.2 25.8 
Pell Grant Eligibility 

Offered Pell 74.4 25.6 

Not Offered Pell 73.4 26.6 
SAT Math Score*** 

Above Median 76.2 23.8 

Below Median 71.4 28.6 
Tuition Residency in First Term 

New York 74.1 25.9 

Other 67.4 32.6 
Total 73.8 26.2 


Note: Graduates employed in NYS refer to graduates with at 
least one quarter with wages above minimum wage threshold 
in any of the six quarters commencing from the quarter of 
graduation. Graduates refer to those completing bachelor's 
degrees within five years of entry, for each of three cohorts 
entering in 2006, 2007, and 2008. For the analysis, those 
identified as “non-resident alien” are excluded, as are 
graduates with cumulative GPA of 2.5 or below who were 
unlikely to have been permitted to take internships. Nineteen 
graduates for whom no Social Security number was available 
for a possible wage match also were excluded from the 
analysis. 


*p<.05, **p<.01, ***p<.001, based on Pearson’s chi-squared 
tests for frequency distribution. 
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‘Table S-3. Time to Employment for Graduates Employed in Table S-4. Quarterly Wages for Graduates Employed in NYS by 
NYS by Internship Experience and Field of Study, Gender, Internship Experience, Field of Study, Gender, Racial/ 
Racial/Ethnic Group, Pell Grant Eligibility, SAT Math Scores, Ethnic Group, Pell Grant Eligibility, SAT Math Scores, 
or Tuition Residency in First Term or Tuition Residency in First Term 
Number of Quarters to first Highest wages in any quarter within 
recorded NYS employment first six quarters from commencement 
Interquartile Standard : p Interquartile s captida 
s a ae Stratum/Grouping Mean Median Range Min Max Deviation 
Stratum/Grouping Mean Median Mode Range Deviation ave 
otal 
Tats Total $9,282 $3,085 $64,199 
Intern-taker 9115 8,183 6,080 3,117 48,929 5,119 
Intern-taker 2.80 2 2 2 1.49 Never-taker 9,341 7,934 6,203 3,085 64,199 5,744 
Never-taker 2.82 2 2 2 1.53 Filed of Study 
Biology 
Field of Study Subtota 7,449 3,199 31,011 
Biology Intern-taker 7,132 6,450 3,361 3,293 15,389 2,960 
ntern-taker 3.02 3 2 2 1.43 Never-taker 7,526 7,116 4,382 3,199 31,011 3,803 
Never-taker 2.99 3 2 2 1.48 Business 
Business cea s1s00 14090 © 54183397 aes .718 
ntern-taker 4 K ‘ i i y 
ntern-taker 237 : 7 7 140 Never-taker 9,920 9,418 6,224 3,154 26,429 4,680 
Never-taker 2.76 2 2 2 1.48 iiealthSclence 
Health Science Subtota 8,162 3,145 64,199 
ntern-taker 2.89 3 3 1 1.42 Intern-taker 8,004 7,286 5,022 3,145 24,441 3,703 
Never-taker 3.27 3 1 3 1.87 Never-taker 8,266 7,038 5,283 3,212 64,199 5,820 
Psychology Psychology 
ntern-taker 2.87 3 1,3 2.25 1.53 Subtota 7,544 3,095 48,929 
Nevar-taker 2.60 2 2 1 1.36 Intern-taker 7,754 6,028 4,732. 3,117 48,929 6,119 
Never-taker 7,482 7,180 3,763 3,095 25,326 3,138 
Gender 
Gender Male 
Male Subtota 9,744 3,142 55,703 
Intern-taker 2.77 2 2 3 1.57 Intern-taker 10,010 9,030 7,106 3,200 48,825 5,340 
Never-taker 2.83 2 2 2 1.56 Never-taker 9,680 8,364 6,728 3,142 55,703 5,747 
Female Female 
Intern-taker 2.81 3 2 1.5 1.44 Subtota 8,878 3,085 64,199 
Never-taker 2.82 3 2 2 1.50 Intern-taker 8,615 7,651 5,346 3,117 48,929 4,927 
Never-taker 8,999 7,626 5,533 3,085 64,199 5,724 
Racial/Ethnic Group pea Sroup 
White Subtota 9,291 3,128 64,199 
ntern-taker 2.59 2 2 1 1.40 Intern-taker 9,394 8,408 5,659 3,128 30,408 4,894 
Never-taker 2.54 2 2 2 1.46 Never-taker 9,260 7,740 6,231 3,142 64,199 5,970 
Asian Asian 
ntern-taker 3.02 3 3 2 1.48 Subtota 10,024 3,117 55,703 
Never-taker 3.15 3 2 ) 1.53 Intern-taker 8,888 7,701 6,075 3,117 25,435 4,495 
Black Never-taker 10,410 8,920 7,255 3,155 55,703 6,280 
ntern-taker 2.76 3 3 2 1.51 Se ae ead 560 ree 
Never-taker 2.82 7 2 7 1.56 Intern-taker 9,353 7,718 6,947 3,200 48,929 7,062 
Hispanic Never-taker 8,382 7,653 4,407 3,338 35,076 4,726 
ntern-taker 2.85 2.5 2 2 1.54 Hispanic 
Never-taker 2.74 2 2 1 1.45 Subtota 8,245 3,141 26,429 
Race Unknown Intern-taker 8997 8,268 6,237 3,540 23,000 4,257 
ntern-taker 2.75 2 2 2 157 Never-taker 7,893 7,555 4,730 3,141 26,429 3,477 
Never-taker 2.89 25 2 2 1.56 Race Unknown 
Subtota 8,928 3,085 50,366 
Pell Grant Eligibility Intern-taker 9,008 8,506 5,480 3,145 48,825 5,601 
Never-taker 8901 7,268 6,167 3,085 50,366 5,373 
Offered Pell as 
Pell Grant Eligibility 
Intern-taker 2.89 3 2 2 1.49 Offered Pell 
Never-taker 2.99 3 2 2 1.56 Subtota 9,638 3,117 64,199 
Not Offered Pell Intern-taker 9,307 8,557 6,117 3,117 48,929 4,918 
Intern-taker 2.72 2 2 1 1.49 Never-taker 9,748 8,370 6,168 3,141 64,199 5,945 
Never-taker 2.68 2 2 1 1.48 Not Offered Pell 
Subtota 8,984 3,085 60,014 
SAT Math Score Intern-taker 8,959 7,698 5,790 3,145 48,825 5,279 
: Never-taker 8,993 7,571 5,910 3,085 60,014 5,545 
AboveiMedian SAT Math Score 
Intern-taker 2.86 3 2 2 1.55 Above Median 
Never-taker 2.85 2 2 2 1.54 Subtota 9,632 3,085 51,572 
Below Median Intern-taker 9,262 8,284 6,113 3,117 48,825 5,320 
Intern-taker 2.74 2 2 2 1.44 Never-taker 9,740 8,230 6,852 3,085 51,572 5,927 
Never-taker 2.80 2 2 2 1.51 Below Median 
Note: Graduates employed in NYS refer to graduates with at least one quarter Subtotal 8,923 3i095 64,199 
with wages above minimum wage threshold in any of the six quarters Intern-taker B98? | 6028 eOae 31128 paced nee 
: i Never-taker 8896 7,592 5,575 3,095 64,199 5,503 
commencing from the quarter of graduation. Graduates refer to those eee SS. SS SaaS 
‘ pees Note: Graduates employed in NYS refer to graduates with at least one quarter with wages above 
completing bachelor's degrees within five years of entry, for each of three a ' - : f 
anfae i . pi minimum wage threshold in any of the six quarters commencing from the quarter of graduation. 
cohorts entering in 2006, 2007, and 2008. For the analysis, those identified as Graduates refer to those completing bachelor's degrees within five years of entry, for each of 
“non-resident alien” are excluded, as are graduates with cumulative GPA of 2.5 three cohorts entering in 2006, 2007, and 2008. For the analysis, those identified as “non-resident 
or below who were unlikely to have been permitted to take internships. alien” are excluded, as are graduates with cumulative GPA of 2.5 or below who were unlikely to 
Nineteen graduates for whom no Social Security number was available for a have been permitted to take internships. Nineteen graduates for whom no Social Security 
possible wage match also were excluded from the analysis. number was available for a possible wage match also were excluded from the analysis. 
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Table S-5. Proportions of Graduates Employed in NYS by Internship Experience and Field of Study, 
Gender, Racial/Ethnic Group, Pell Grant Eligibility, SAT Math Scores, or Tuition Residency in First Term 


Intern-taker Never-taker 
Percent Percent 
Number of Employedin Numberof Employedin Difference in Standard 
Stratum/Grouping Graduates NYS Graduates NYS _ Proportions Error 
Total 1,154 58.9 3,334 57.6 1.3 1.7 
Field of Study 
Biology 104 51.0 426 46.2 4.8 5:5 
Business 126 77.0 271 67.9 5.1 ** 4.7 
Health Science 261 59.4 398 53.3 6:1 * 3.9 
Psychology 118 57.6 401 62.6 -5.0 5.2 
Other 545 56.3 1,838 58.5 -2.2 5.7 
Gender 
Male 421 58.0 1,733 55.8 22 2.7 
Female 733 59.5 1,601 59.5 0.0 2:2 
Racial/Ethnic Group 
White 333 57.7 1,094 60.9 -3.2 3.1 
Asian 363 54.5 1,068 52.1 2.4 3.0 
Black 95 70.5 187 66.8 3.7 5.8 
Hispanic 103 75:7 220 70.5 5.2 5.2 
Race Unknown 259 56.0 757 54.4 1.6 3.6 
Pell Grant Eligibility 
Offered Pell 495 61.8 1,487 59.7 2.1 2.5 
Not Offered Pell 659 56.8 1,847 55.9 0.9 22 
SAT Math Score 
Above Median 558 56.5 1,902 53.1 3.4 * 2.4 
Below Median 596 61.2 1,432 63.5 -2.3 2.4 
Tuition Residency in First Term 
New York 1,045 62.1 3,095 59.9 2.2 1.7 
Other 109 28.4 239 26.8 1.6 5.2 
Note: Graduates employed in NYS refer to graduates with at least one quarter with wages above minimum wage 
threshold in any of the six quarters commencing from the quarter of graduation. Graduates refer to those completing 
bachelor's degrees within five years of entry, for each of three cohorts entering in 2006, 2007, and 2008. For the 
analysis, those identified as “non-resident alien” are excluded, as are graduates with cumulative GPA of 2.5 or below 
who were unlikely to have been permitted to take internships. Nineteen graduates for whom no Social Security 
number was available for a possible wage match also were excluded from the analysis. 
*p<.05, **p<.01, ***p<.001, based on one-tail Z-Test of significance, two sample test. 
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Table S-6. Proportions of Graduates Employed in NYS by Internship 


Experience and Field of Study, Gender, Racial/Ethnic Group, Pell Grant 
Eligibility, SAT Math Scores, or Tuition Residency in First Term 


Intern-taker Never-taker 
Percent Percent 
Number of Employed in Number of Employedin Differencein Standard 
Stratum/Grouping Graduates NYS Graduates NYS Proportions Error 
Total 1,154 58.9 3,334 57.6 1.3 1:7 
Field of Study and Gender 
Biology 
Subtotal 104 51.0 426 46.2 48 5.5 
Male 44 43.2 233 44.6 -14 8.1 
Female 60 56.7 193 48.2 8.5 7.3 
Business 
Subtotal 126 77.0 271 67.9 Che 47 
Male 62 83.9 165 66.7 A7,.2°99* 5.9 
Female 64 70.3 106 69.8 0.5 7.2 
Health Science 
Subtotal 261 59.4 398 53.3 6.1 * 3.9 
Male 83 55.4 155 47.1 8.3 6.8 
Female 178 61.2 243 57.2 4.0 48 
Psychology 
Subtotal 118 57.6 401 62.6 -5.0 5:2 
Male NA NA 
Female 98 62.2 272 64.0 -18 Wi 
Field of Study and Racial/Ethnic Group 
Biology 
Subtotal 104 51.0 426 46.2 48 5:5 
White 23 52.2 114 43.9 8.3 11.4 
Asian 50 46.0 161 46.6 -0.6 8.1 
Black NA NA 
Hispanic NA NA 
Race Unknown NA NA 
Business 
Subtotal 126 77.0 271 67.9 9.1 ** 47 
White 32 65.6 93 73.1 75 9.6 
Asian 50 74.0 92 62.0 12.0 * 8.0 
Black NA NA 
Hispanic NA NA 
Race Unknown 26 80.8 52 65.4 15.4 * 10.2 
Health Science 
Subtotal 261 59.4 398 53:3 6.1 * 3.9 
White 45 60.0 86 53.5 6.5 91 
Asian 108 57.4 175 44.6 12.8:** 6.1 
Black 32 65.6 40 57.5 8.1 115 
Hispanic 17 64.7 32 81.3 -16.6 * 13.5 
Race Unknown 58 58.6 64 60.9 -2.3 8.9 
Psychology 
Subtotal 118 57.6 401 62.6 -5.0 5:2 
White 33 48.5 147 65.3 -16.8 ** 9.5 
Asian 29 SL7 93 53.8 21 10.6 
Black NA NA 
Hispanic 16 75.0 40 80.0 -5.0 12.5 
Race Unknown 34 55.9 97 56.7 -0.8 9.9 
Field of Study and Pell Grant Eligiblity 
Biology 
Subtotal 104 51.0 426 46.2 48 5:5 
Offered Pell 44 61.4 188 50.5 10.9 * 8.2 
Not Offered Pell 60 43.3 238 42.9 0.4 7.2 
Business 
Subtotal 126 77.0 271 67.9 Cad 47 
Offered Pell 62 85.5 136 69.9 15.6" 6.0 
Not Offered Pell 64 68.8 135 65.9 2.9 7.1 
Health Science 
Subtotal 261 59.4 398 53:3. 6.1 * 3.9 
Offered Pell 136 60.3 202 59.4 0.9 5.4 
Not Offered Pell 125 58.4 196 46.9 21.5'%% 5.7 
Psychology 
Subtotal 118 57.6 401 62.6 -5.0 5.2 
Offered Pell 5S) 58.2 165 66.1 73 7.6 
Not Offered Pell 63 Ey fk 236 60.2 -3.1 7.0 
Field of Study and SAT Math Score 
Biology 
Subtotal 104 51.0 426 46.2 48 5.5 
Above Median 60 bay: 291 44,3 7A VA 
Below Median 44 50.0 135 50.4 -0.4 8.7 
Business 
Subtotal 126 77.0 271 67.9 9.1 ** 47 
Above Median 73 80.8 142 60.6 20:2 *** 6.2 
Below Median 53. ALT 129 76.0 4.3 72 
Health Science 
Subtotal 261 59.4 398 53.3 6.1:* 3.9 
Above Median 109 58.7 192 49.0 9.7°* 5.9 
Below Median 152 59.9 206 $2:3 2.6 Pye} 
Psychology 
Subtotal 118 57.6 401 62.6 -5.0 Pie 
Above Median 41 61.0 171 59.1 1.9 8.5 
Below Median ah 55.8 230 65.2 9.4 * 6.5 
Note: Graduates employed in NYS refer to graduates with at least one quarter with wages above minimum wage threshold in any 
of the six quarters commencing from the quarter of graduation. Graduates refer to those completing bachelor's degrees within 
five years of entry, for each of three cohorts entering in 2006, 2007, and 2008. For the analysis, those identified as “non-resident 
alien” are excluded, as are graduates with cumulative GPA of 2.5 or below who were unlikely to have been permitted to take 
internships. Nineteen graduates for whom no Social Security number was available for a possible wage match also were excluded 
from the analysis. 
*p<.05, **p<.01, ***p<.001, based on one-tail Z-Test of significance, two sample test. 
NA Less than ten graduates with matched NYDOL wage records. 
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Table S-7. Differences in Time to Employment and Differences in Quarterly Wages for Graduates Employed in NYS by Internship 
Experience, Field of Study, Gender, Racial/Ethnic Group, Pell Grant Eligibility, SAT Math Scores, or Tuition Residency in First Term 


Quarters to First NYS Quarterly Wages 
Employment Quarterly Wages (without outlier) 
Difference in Standard Difference in Standard Difference in Standard 
Stratum/Grouping Means Error Means Error Means Error 
Total -0.029 0.068 -226 237 -256 228 
Field of Study 
Biology 0.029 0.229 -394 492 -274 517 
Business -0.390 ** 0.183 1,980 *** 678 1,686 *** 564 
Health Science -0.383 ** 0.173 -262 502 -101 413 
Psychology 0.268 * 0.192 272 774 -271 470 
Gender 
Male -0.056 0.112 330 407 217 354 
Female -0.011 0.086 -384 301 -419 * 281 
Racial/Ethnic Group 
White 0.047 0.119 134 423 107 402 
Asian -0.132 0.126 -1,522 *** 418 -1,524 *** 402 
Black -0.055 0.235 971 967 587 738 
Hispanic 0.111 0.206 1,104 ** 560 1,043 ** 522 
Race Unknown -0.140 0.152 107 527 -70 473 
Pell Grant Eligibility 
Offered Pell -0.097 0.103 -441 346 -510 * 315 
Not Offered Pell 0.034 0.090 -34 331 -92 302 
SAT Math Score 
Above Median 0.009 0.100 -478 * 354 -563 ** 329 
Below Median -0.054 0.093 93 318 44 292 
Note: Graduates employed in NYS refer to graduates with at least one quarter with wages above minimum wage threshold in 
any of the six quarters commencing from the quarter of graduation. Graduates refer to those completing bachelor's degrees 
within five years of entry, for each of three cohorts entering in 2006, 2007, and 2008. For the analysis, those identified as “non- 
resident alien” are excluded, as are graduates with cumulative GPA of 2.5 or below who were unlikely to have been permitted to 
take internships. Nineteen graduates for whom no Social Security number was available for a possible wage match also were 
excluded from the analysis. 
Outlier is defined as a group maximum value that is larger than the group mean plus three standard deviations or a group 
minimum value that is smaller than the group mean minus three standard deviations. 
*p<.05, **p<.01, ***p<.001, based on one-tail t-Test of significance, two sample test. 


Rockefeller Institute Page 24 www.rockinst.org 


Higher Educat Supplementary Report on Assessing Labor Market Outcomes: A Pilot Study 


Table S-8. Differences in Time to Employment and Differences in Quarterly Wages for 


Graduates Employed in NYS by Internship Experience, Field of Study, Gender, Racial/ 
Ethnic Group, Pell Grant Eligibility, SAT Math Scores, or Tuition Residency in First Term 


Quarters to First Quarterly Wages 
NYS Employment Quarterly Wages (without outlier) 
Difference in Standard Difference in Standard Difference in Standard 
Stratum/Grouping Means Error Means Error Means Error 
Field of Study and Gender 
Biology 
Male 0.545 * 0.365 -711 773 -485 741 
Female -0.265 0.300 -94 593 
Business 
Male -0.537 ** 0.264 2,483 ** 1,075 1,920 *** 788 
Female -0.186 0.248 1,386 ** 818 
Health Science 
Male -0.789 ** 0.338 652 864 893 816 
Female -0.164 0.203 -655 632 -404 459 
Psychology 
Male NA NA NA NA NA NA 
Female 0.171 0.215 488 857 -91 514 
Field of Study and Racial/Ethnic Group 
Biology 
White 0.347 0.494 -1,392 1,083 -941 992 
Asian -0.201 0.329 -67 758 91 707 
Black NA NA NA NA NA NA 
Hispanic NA NA NA NA NA NA 
Race Unknown NA NA NA NA NA NA 
Business 
White -0.467 0.367 1,430 1,190 
Asian -0.354 0.307 1,043 * 770 
Black NA NA NA NA NA NA 
Hispanic NA NA NA NA NA NA 
Race Unknown -0.429 0.452 3,463 * 2,150 NA NA 
Health Science 
White -0.266 0.388 183 1,695 846 971 
Asian -0.604 ** 0.283 -996 * 686 -781 670 
Black -0.481 0.491 -233 1,572 984 1,005 
Hispanic -0.083 0.576 883 1,110 
Race Unknown -0.410 0.407 474 863 915 737 
Psychology 
White 0.021 0.369 -610 803 -509 768 
Asian -0.006 0.430 -603 1,042 -226 793 
Black NA NA NA NA NA NA 
Hispanic 0.417 0.511 354 1,110 687 1,206 
Race Unknown 0.206 0.381 102 1135 -473 775 
Field of Study and Pell Grant Eligiblity 
Biology 
Offered Pell 0.111 0.319 -440 668 -320 632 
Not Offered Pe’ -0.057 0.333 -361 730 -127 797 
Business 
Offered Pell -0.433 ** 0.242 1,951.4" 714 2,107 *** 686 
Not Offered Pel -0.357 0.281 2,050 * 1,248 1,384 * 907 
Health Science 
Offered Pell -0.694 *** 0.233 -662 756 -186 591 
Not Offered Pe! 0.002 0.262 297 591 256 516 
Psychology 
Offered Pell 0.262 0.332 1,769 1,502 662 808 
Not Offered Pe! 0.264 0.260 -1,096 ** 574 -1,226 *** 460 
Field of Study and SAT Math Score 
Biology 
Above Median -0.183 0.300 -760 677 -578 706 
Below Median 0.359 0.360 203 748 
Business 
Above Median -0.408 * 0.268 2,285 *** 936 1,650 *** 684 
Below Median -0.421 * 0.256 1,640 ** 985 1,808 ** 952 
Health Science 
Above Median -0.256 0.270 265 720 299 613 
Below Median -0.472 ** 0.228 -671 707 -183 513 
Psychology 
Above Median 0.427 * 0.324 -1,236 ** 703 -1,057 ** 607 
Below Median -0.054 0.093 1,143 1,159 271 660 
Note: Graduates employed in NYS refer to graduates with at least one quarter with wages above minimum wage threshold in any of the 
six quarters commencing from the quarter of graduation. Graduates refer to those completing bachelor's degrees within five years of 
entry, for each of three cohorts entering in 2006, 2007, and 2008. For the analysis, those identified as “non-resident alien” are excluded, 
as are graduates with cumulative GPA of 2.5 or below who were unlikely to have been permitted to take internships. Nineteen 
graduates for whom no Social Security number was available for a possible wage match also were excluded from the analysis. 
Outlier is defined as a group maximum value that is larger than the group mean plus three standard deviations or a group minimum 
value that is smaller than the group mean minus three standard deviations. If no outlier is present, there is no entry in the rows 
concerned. 
*p<.05, **p<.01, ***p<.001, based on one-tail t-Test of significance, two sample test. 
NA Less than ten graduates with matched NYDOL wage records. 
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Zimpher Aims to Boost Completion,” Press Release, August 26, 2015, 
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Information from other systems/states as well as for SUNY suggest that as many as half of graduates from a 
campus may not appear on NYSDOL wage records within the first four to six quarters following 
graduation. For CUNY, more than 80 percent of associate and bachelor’s graduates were employed (and 
showed up on wage records) during the period 2003 to 2010. See Colin C. Chellman, “Measuring 
Employment Outcomes at CUNY,” Presentation at the Center for Analysis of Postsecondary Education and 
Employment (CAPSEE) Conference, Washington, DC, September 19, 2014, 

http:/ /capseecenter.org/wp-content/uploads/2014/09/3D-Chellman-CAPSEE-091914.pdf. The proportion 
of graduates with linked UI wage records in this pilot study of one SUNY campus is relatively lower, owing 
to the shorter period covered and attention to the bachelor’s level. Further, data from the pilot study show 
matches with wage records for 60 percent of those with NYS residency for tuition fee purposes compared to 
28 percent matches for non-NYS residents. So, nonresidents returning to their “home” state for employment 
also could account, in part, for a lower share of wage record matches. The share is lower, also, owing to 
relatively high rates of continuation into graduate study. Using academic records, Colorado locates 21 
percent of its public college and university graduates in postgraduate study in the state within six months of 
graduation and, of this group, 8 percent come up on wage record files with incomes above the minimum 
wage threshold. See Schneider, Education Pays in Colorado. 


But, see, Bransberger, A Glimpse Beyond State Lines, and Chelman, “Measuring Employment Outcomes at 
CUNY,’ especially for tracking graduates across state borders and addressing multiple wage records when, 
for example, individuals receive compensation from more than one employer. 
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On this point, see Schneider, Education Pays in Colorado. For identified fields, differences by college or 
university in Colorado are commonly modest in the first year, but show much more marked differences by 
year ten. 


See, Mark Schneider, Measuring the Economic Success of College Graduates: Lessons from the Field (Washington, 
DC: Education Policy Center at American Institutes of Research, June 2014), 
http://www.air.org/sites/default/files/ downloads/report/ Measuring % 20the % 20Economic % 20Success %2 
Oof% 20College % 20Graduates_Mark%20Schneider.pdf; Washington State Education Research & Data 
Center, Employment Data Handbook: A Guide for Incorporating Employment Information from a State 
Unemployment Insurance (UI) Program into a P-20 Longitudinal Data System, EDRC Technical Report 2012-01 
(Olympia: Education Research & Data Center, 2012). 


See, e.g., Schneider, Measuring the Economic Success of College Graduates and Education Pays in Colorado. For 
the most recent Colorado study, earnings are summed for four consecutive quarters and evaluated against 
the wage threshold. For a previous study for the state, a minimum wage threshold was applied for each 
quarter. Simple comparisons of medians, evaluated at annual equivalents, show a similar order of 
magnitude for Colorado graduates and graduates in this pilot study, respectively, $33,091 and $32,723 
(internship taker) and $31,736 (non-taker), all in 2014 dollars. Analyses undertaken in Texas using UI wage 
records obtain measures of earnings in four consecutive quarters and adopt a minimum wage and full-time 
employment threshold for the calendar year. Applying such a threshold, the median salary within one year 
of graduation for Texas graduates is higher, at $44,100 (in 2013 dollars). For a detailed description of the 
Texas analysis and data, see the seekUT website, University of Texas System, Austin, n.d., 
http://www.utsystem.edu/seekut/about_the data.html. 


A separate threshold is applied to the quarter in which earnings were recorded, as follows: For Q1 2008 
through Q2 2009, $3,003; for Q2 2009 through OQ4 2013, $3,045; for Q1 2014 through O4 2014, $3,360. Each 
threshold is calculated as the NYS minimum wage in effect during the period in question times thirty-five 
hours per week over twelve weeks. The minimum wage figures comes from NYS Department of Labor, 
“History of the Hourly Minimum Wage,” Last Updated December 31, 2015, 
http://www.labor.ny.gov/stats/minimum_wage.asp. 


Due to the large number of “paired” groups, we may have a multiple comparisons problem. We chose not 
to apply corrections because the aim of this study is to explore whether there are possible effects of 
internships, not to generate evidence that we have identified such effects. Therefore, we are giving priority 
to reducing the likelihood of Type II error, i.e. the failure to reject a false null hypothesis. While both the 
very strict Bonferroni correction and the more preferred Benjamini-Hochberg procedure (which controls for 
the false discovery rate) could help address the Type I error introduced by multiple comparisons, it is at the 
expense of increasing the possibility of the Type II error. Moreover, the corrections are more applicable to 
analyses of a series of independent tests. Arguably, our eighty-eight “pairs” are not independent; instead, 
most of them are nested. 


See, for example, Schneider, Measuring the Economic Success of College Graduates and Education Pays in 
Colorado; Bransberger, A Glimpse Beyond State Lines. 


We undertook a separate, simple analysis to uncover evidence that extreme values might have affected the 
simple tests of significance of differences in employment outcomes by internship taking. In this 
supplementary analysis, with results provided in Tables S.7 and S.8, we eliminated any maximum or 
minimum value that was, respectively, three standard deviations above the mean and three standard 
deviations below the mean, in any group, adjusted group means and standard deviations accordingly, and 
carried out the statistical tests again for each “paired” group. The results largely were the same. 


Following Angrist, it is possible to use grouped data in multivariate analyses. However, in the present 
study, we lack a sufficient number of groups to meet coverage requirements for applying such techniques. 
Angrist had data for 64.5 percent of all possible cells, covering 98 percent of the observations in his sample. 
By comparison, we have data for 4.0 percent of all possible cells (at the deepest level of stratification) 
accounting for 10.7 percent of the observations in the sample drawn for the pilot study. See Joshua D. 
Angrist, “Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on 
Military Applicant,” Econometrica 66, 2 (1998): 249-88. 


Also, on analyses with group data, see Endnotes 22 and 25. 
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